DEEP LEARNING CA1 PART B: CONVOLUTIONAL NEURAL NETWORK 2

Yek Yi Wei
P2107631
DAAA/FT/2B/03

Import Libraries

Background Research

CIFAR100 images are originally 32x32. For modern CNN architectures (such as Resnet18) we have to upscale these images to proper size (256x256). Doing this upscaling during data loading can cause CPU bottleneck. So here I have preporcessed the images to 256x256 so that you can directly load them.

Check GPU

EDA

LOAD TRAINING DATASET

Distribution of all the classes

DATA PREPROCESSING

-Normalize the data
-Scaled the values to a range of 0 to 1 before feeding to the neural network model. 
-Divide the values by 255. 
-The training set and the testing set are preprocessed in the same way

Prepare ImageDataGenerator for data augmentation

Adding the Augmented data to the default data

Plot Learning Curves

Plot accuracy on the training and validation datasets over training epochs.

Plot loss on the training and validation datasets over training epochs.

Early Stopping

Early Stopping is a callback that allows you to specify the performance measure to monitor, the trigger, and once triggered, it will stop the training process.

Base Model

Without data Augmentation

Model severely overfits the train data which cauzes the test data to be very low

With data Augmentation

Improved Model 1

Without data Augmentation

With data Augmentation

Using the Augmented data

Using the Augmented data and the default data

Improved Model 2

Without data Augmentation

With data Augmentation

Using the Augmented data

Model severely overfits the train data which cauzes the test data to be very low

Using the Augmented data and the default data

Hyperparameter Search (Keras Tuner: Random Search)

Tune the best model, which is improved model 2 using the Augmented data and the default data, with the higher validation accuracy and lower validation loss (With no overfitting).

-Parameters input dropout to between 0.1 and 0.2 because dropping the input data can adversely affect the training.
-Parameters intermediate dropout to 0.5 and below because 0.5 is ideal for large datasets and >0.5 is not advised it may cull more connections without boosting the regularization.

Using RandomSearch in keras tuner

Scores show lower validation accuracy and higher validation loss

Predictions on the test set

Now, let’s use our best model before and after the hypertuning to make predictions on the test set and check our results.

Improved Model 2 Prediction

Correct prediction labels are blue and incorrect prediction labels are red. The number gives the percent (out of 100) for the predicted label.

Improved Model 2 Prediction AFTER Hypertuning

Save my models

Improved Model 1

Save my best model (Improved Model 2)

Tuned Improved Model 2